WIP: Surface filter-promoted unraisable warnings directly (#14263)#14499
Draft
paulzuradzki wants to merge 7 commits into
Draft
WIP: Surface filter-promoted unraisable warnings directly (#14263)#14499paulzuradzki wants to merge 7 commits into
paulzuradzki wants to merge 7 commits into
Conversation
d7db869 to
476e4f5
Compare
filterwarnings = error::ResourceWarning does not fail tests that leak resources through a reference cycle. collect_unraisable wraps the captured ResourceWarning in PytestUnraisableExceptionWarning, a class the user has no filter for, so the run exits 0. This test pins that contract: on a refcycle-leaking test with error::ResourceWarning configured, pytest should exit non-zero and the output should show the inner ResourceWarning rather than the wrapping PytestUnraisableExceptionWarning. Fails at this commit; the next commit ships the fix that turns it green. Refs pytest-dev#14263.
When sys.unraisablehook captures a Warning subclass instance and the user has an active ``error::<that class>`` filter, raise the original warning rather than wrapping in PytestUnraisableExceptionWarning. The wrap path remains for any case where no matching error filter is set, so suites that don't use ``error::<warning>`` filters see no change. Filter matching is approximate: category only, not message/module/lineno. The check errs toward false negatives, never false positives. The regression test added in the previous commit now passes. Additional coverage: - test_refcycle_userwarning_filter: locks the contract for a non-builtin Warning subclass. - test_unraisable_warning_without_filter_still_wraps: scope guard. A Warning raised from __del__ without a matching error filter must still be wrapped, not raised directly. - test_unraisable_warning_filter_add_note_dedups: covers the duplicate- note guard in the unwrap path for singleton/cached Warning instances. Tightens the ``errors`` list type from list[Exception] to list[Warning | RuntimeError]. Adds Paul Zuradzki to AUTHORS. Notes in test_create_task_raises_unraisable_warning_filter that the propagated class is now bare RuntimeWarning rather than the wrapping PytestUnraisableExceptionWarning (because -Werror activates the new unwrap path). Closes pytest-dev#14263.
476e4f5 to
4cf46d4
Compare
FuzzysTodd
approved these changes
May 20, 2026
FuzzysTodd
left a comment
There was a problem hiding this comment.
pytest_unconfigure fires before _cleanup_stack.close(), so warning
filters managed via the cleanup stack (the warnings plugin's
catch_warnings context, in particular) are guaranteed active when GC
The previous arrangement works because PR pytest-dev#13057 (Dec 2024) reordered default_plugins so the unraisableexception cleanup pops before the warnings plugin's catch_warnings exit under LIFO. Correctness then relies on registration order; any plugin that registers a cleanup later than unraisableexception's and resets warnings.filters revives the original bug. The test uses a conftest with @hookimpl(trylast=True) to register a warnings.resetwarnings cleanup after the built-in plugins. Under LIFO, that cleanup pops first; warnings.filters is empty by the time unraisableexception runs gc.collect() and the leaking finalizer fires. warnings.warn then emits the ResourceWarning silently, since no filter promotes it, and the suite exits 0. Fails at this commit. The next commit moves gc.collect() and unraisable processing into pytest_unconfigure, which runs before _cleanup_stack.close(); the test then passes regardless of plugin registration order. Refs pytest-dev#14263.
Register only the hook-restore + stash-cleanup as the config.add_cleanup callback. Move the GC pump and collect_unraisable call into a new pytest_unconfigure(config) hook. pytest_unconfigure fires before _cleanup_stack.close(), so warning filters managed via the cleanup stack (the warnings plugin's catch_warnings context, in particular) are guaranteed active when GC runs. This decouples the unraisable step from plugin registration order in default_plugins. The previous arrangement worked only because of LIFO ordering on _cleanup_stack; pytest-dev#13057 (Dec 2024) reordered default_plugins to make that ordering correct, but the structural fragility remained. No observable behavior change. The 141 existing tests in test_unraisableexception + test_warnings + test_recwarn + test_threadexception still pass. The previous commit's test_refcycle_resource_warning_filter continues to fail on main and pass here. This is the structural side of the issue's two proposed fixes; the user-visible side shipped in the previous commit.
The previous commit moved gc.collect() and unraisable processing from the config.add_cleanup callback into pytest_unconfigure. pytest calls pytest_unconfigure for every plugin during shutdown, regardless of whether that plugin's pytest_configure completed. When another plugin's pytest_configure raises pytest.UsageError, pluggy stops calling the remaining configure hooks; unraisableexception's pytest_configure never runs and its stash key stays unset. Without a presence check at the top of pytest_unconfigure, the unraisable plugin subscripts config.stash[unraisable_exceptions] on an unset key, hits KeyError, and pytest reports INTERNALERROR instead of USAGE_ERROR. The user sees a pytest-internal traceback instead of the plain "ERROR: <their bad config>" message. The test installs a conftest whose pytest_configure raises UsageError, then asserts the run exits with USAGE_ERROR and that no INTERNALERROR or KeyError appears in stderr. Fails at this commit; the next commit adds the guard. Refs pytest-dev#14263.
When another plugin's pytest_configure raises (e.g. pytest.UsageError in testing/acceptance_test.py::test_config_error), pluggy skips remaining configure hooks. unraisableexception.pytest_configure never runs, config.stash[unraisable_exceptions] is never set. The previous config.add_cleanup callback wasn't registered in that case either, so cleanup was a no-op. The pytest_unconfigure hook introduced in the previous commit ran unconditionally and hit KeyError on the unset stash, surfacing as INTERNAL_ERROR where pytest should exit with USAGE_ERROR. Guard with a stash-presence check at the top of pytest_unconfigure. test_config_error catches the regression direction.
Drop "drain" in favor of literal language. The pytest_unconfigure comment now states what the hook depends on (filters managed via the cleanup stack, garbage-collected finalizers) rather than naming the previous mechanism. The guard comment says "the queue stash was never set" instead of the metaphorical "nothing to drain."
63a5109 to
56f71b7
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #14263.
filterwarnings = error::ResourceWarningdoes not fail tests that leak resources through a reference cycle.collect_unraisablewraps the captured warning inPytestUnraisableExceptionWarning, a class the user has not filtered.collect_unraisablenow checkswarnings.filtersbefore wrapping. When an activeerrorfilter targets a class that the captured warning is an instance of,collect_unraisablere-raises that warning directly; the resulting exception fails the test.Visual
Behavior on
main: this test passes despite user configuring for ResourceWarning to trigger an error. Picturedtest_leak.pytriggers ResourceWarning.After PR: raises error as expected
Scope
collect_unraisablechooses between two branches when it processes a queued unraisable:error::<class>filter fails the test as written. Applies only when an activeerror::<class>filter matches that class.PytestUnraisableExceptionWarningand emit it viawarnings.warn. A__del__that raises aWarningwithout a matchingerror::<class>filter still hits this branch.Suites without
error::<warning>filters see no change.The unwrap-branch filter check is
action == "error"plusissubclass(warning_class, filter_category). It does not re-run Python'swarningsmodule's full match logic, which additionally tests the filter'smessage_regex,module_regex, andlinenofields. Consequence: for users with narrowly-scoped filters likeerror:some_msg:ResourceWarning::42, this branch can fail a test on aResourceWarningwhose message wouldn't have matched the regex. For broad filters likeerror::ResourceWarningthe behavior matches user intent.Branch structure
Three red→green commit pairs plus a comment-cleanup commit. Each red commit fails the named test; the next commit makes it pass.
92c7681a2(red) →4cf46d4f9(fix):test_refcycle_resource_warning_filterexercises the user-visible wrap-vs-unwrap behavior covered in Summary.21d334ad1(red) →839a20a0c(fix): the move-GC hardening from Fix session-end gc in unraisableexception plugin to respect warning filters #14273.test_unraisable_decouples_from_cleanup_stack_orderuses@hookimpl(trylast=True)to force a LIFO cleanup-stack order that PR apply warnings filter as soon as possible, and remove it as late as possible #13057 incidentally avoided. The fix movesgc.collect()and queue processing intopytest_unconfigure, which runs before_cleanup_stack.close().dead24a90(red) →ef0d16180(fix):test_pytest_unconfigure_survives_failed_pytest_configureraisesUsageErrorfrom a conftest'spytest_configure, which exposed aKeyErrorafter step 2 moved the GC call out ofconfig.add_cleanup. The fix adds a stash-presence check at the top ofpytest_unconfigure.Cleanup:
56f71b785tightens the new comments inunraisableexception.py.Relationship to #14273
#14273 attempted approach 2 from the issue (move GC to
pytest_unconfigure). Maintainers closed it unmerged: #13057 (Dec 2024) had already shipped approach 1, so the move-GC change alone produced no observable behavior difference and no fails-then-passes regression test was possible. @Zac-HD asked: "we'd need to see a regression test which demonstrates that the unraisable hook is now subject to warnings."Step 2 in the branch structure addresses that question. The red test forces the LIFO ordering that #13057 incidentally avoided, demonstrating the structural fragility was load-bearing on plugin registration order rather than benign.
Testing
Reproducer
My adapted version of the original gist from the issue. The original references an undefined
stderr_linesin itsrun_scenariohelper; the adapted version applies a one-line fix.For a one-file check, save this as
test_leak.py:Then run
pytest -W error::ResourceWarning test_leak.py. The-Wflag drives the same code path as thepytest.inifilterwarningssetting. Onmainthe test passes (bug); on this branch it exits 1 with the innerResourceWarningraised directly, noPytestUnraisableExceptionWarningwrapper.If you run from inside the pytest source tree, the repo's
pyproject.toml[tool.pytest]filterwarnings = ['error', ...]promotes every warning to an error. Comment out the bare'error'entry, or run the snippet from outside the repo.mainmain, install (pip install -e .), run the adapted reproducer. Scenario 1 exits 0 (bug confirmed). Scenario 2 exits 1.ResourceWarningraised. Scenario 2 still exits 1.92c7681a2,21d334ad1,dead24a90); the named test fails. Check out the next commit; it passes.External suite tests
(time-permitting; I don't think this needs to block review)
Warningfrom__del__without a matchingerrorfilter against this branch.error::DeprecationWarningorerror::UserWarningfilters against this branch. Nothing previously-passing should now fail.AI disclosure
Side note: open to feedback to pull some of the PR comments into the code if helpful
Changelog
changelog/14263.bugfix.rst.